On one hand, the knowledge learned from public data is implicitly transferred into the private model.

The hyperparameter settings for Sageflow are shown in Table 1. Table 2. Backdoor attack: The hyperparameter details are shown in Table 4. Table 4: Hyperparameters for Sageflow with both stragglers and adversaries, under backdoor attackDataset γ λ δ E We specify these values in Table 5. The local batch size is set to 64. Figure 1 shows the performance under the no-scaled backdoor attack with only adversaries (no stragglers). Figure 1 shows the case with both stragglers and adversaries. Some additional experiments were conducted under model poisoning with the scale factor 10. Figure 1 The loss associated with a poisoned device increases if we increase the scale factor from 0.1 to 10. Sageflow but also Zeno+ can effectively defend against the attacks with only adversaries.

While federated learning (FL) allows efficient model training with local data at edge devices, among major issues still to be resolved are: slow devices known as stragglers and malicious attacks launched by adversaries.

We first present an algorithmic framework that unifies a long line of iterative algorithms in the literature. Under this framework, we propose two new methods.

Due to statistical lower bounds on the learnability of many function classes under privacy constraints, there has been recent interest in leveraging public data to improve the performance of private learning algorithms. In this model, algorithms must always guarantee differential privacy with respect to the private samples while also ensuring learning guarantees when the private data distribution is sufficiently close to that of the public data. Previous work has demonstrated that when sufficient public, unlabelled data is available, private learning can be made statistically tractable, but the resulting algorithms have all been computationally inefficient. In this work, we present the first computationally efficient, algorithms to provably leverage public data to learn privately whenever a function class is learnable non-privately, where our notion of computational efficiency is with respect to the number of calls to an optimization oracle for the function class. In addition to this general result, we provide specialized algorithms with improved sample complexities in the special cases when the function class is convex or when the task is binary classification.

The superior performance of large foundation models can be attributed to the use of massive amounts of high-quality data.

We propose conducting locally differentially private (LDP) estimation with the aid of a small amount of public data to enhance the performance of private estimation. Specifically, we introduce an efficient algorithm called Locally differentially Private Decision Tree (LPDT) for LDP regression. We first use the public data to grow a decision tree partition and then fit an estimator according to the partition privately. From a theoretical perspective, we show that LPDT is $\varepsilon$-LDP and has a mini-max optimal convergence rate under a mild assumption of similarity between public and private data, whereas the lower bound of the convergence rate of LPDT without public data is strictly slower, which implies that the public data helps to improve the convergence rates of LDP estimation. We conduct experiments on both synthetic and real-world data to demonstrate the superior performance of LPDT compared with other state-of-the-art LDP regression methods. Moreover, we show that LPDT remains effective despite considerable disparities between public and private data.